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Abstract 

Analyzing  people,  equipment,  organizations  and  their  activities  over  time  is  increasingly 
a  key  concern  for  command  and  control.  The  objective  of  our  research  has  been  to 
develop  a  method  to  capture  and  visualize  the  spatial  and  inter-connectedness  of  this 
information  over  time  within  a  single,  highly  interactive  picture.  A  novel  visualization 
technique  for  displaying  and  tracking  events,  people,  and  equipment  within  a  combined 
temporal  and  geospatial  display  has  been  developed  into  a  demonstratable  prototype 
called  GeoTime  in  order  to  determine  potential  utility.  The  focus  of  this  paper  is  on  land 
force  commanders  and  staff.  However,  we  believe  the  concept  is  applicable  to  a  wide 
variety  of  government  and  business  analysis  tasks. 


Introduction 

Understanding  people,  equipment,  organizations  and  their  activities  over  time  is 
increasingly  a  key  concern  for  command  and  control  in  any  deployment.  In  situations  of 
asymmetric  warfare,  stability  and  support  operations  and  peace  support  operations, 
tracking  relatively  small  and  seemingly  unconnected  events  over  time  becomes  a  means 
for  gaining  insight  into  enemy  or  insurgent  behavior.  Local  cultures,  characters, 
organizations  and  their  behaviors  play  an  important  part  in  planning  and  mission 
execution.  Small  attacks  come  from  unknown  assailants  over  long  periods  of  time, 
making  it  difficult  to  see  patterns.  Relationships  between  events,  suspects,  known 
personalities  and  organizations  are  hard  to  make.  Information  obtained  is  piecemeal  -  a 
puzzle.  The  key  dimensions  of  this  information  are  who,  what,  where,  when,  how,  why 
and  with-whom. 

Visualization  and  interaction  techniques  have  significant  potential  to  improve  the 
productivity  of  complex  human  cognitive  tasks.  They  serve  as  external  mental  aids 
[Tufte,  1990],  When  information  is  presented  visually,  efficient  innate  human 
capabilities  can  be  used  to  perceive  and  process  data.  Information  visualization 
techniques  amplify  cognition  by  increasing  human  mental  resources,  reducing  search 
times,  improving  recognition  of  patterns,  increasing  inference  making,  and  increasing 
monitoring  scope  [Card,  1999],  [Ware,  2000],  These  benefits  translate  into  significant 
system  and  task  related  performance  objectives  [Wright,  Kapler,  2002], 


Background 

Many  visualization  techniques  and  products  for  analyzing  complex  event  interactions 
only  display  information  along  a  single  dimension,  typically  one  of  time,  geography  or  a 
network  connectivity  diagram..  Each  of  these  types  of  visualizations  is  common  and  well 
understood.  For  example  a  time-focused  scheduling  chart  such  as  Microsoft  (MS)  Project 
displays  various  project  events  over  the  single  dimension  of  time,  and  a  Geographic 
Information  System  (GIS)  product,  such  as  MS  MapPoint,  or  ESRI  ArcView,  is  good  for 
showing  events  in  the  single  dimension  of  locations  on  a  map.  There  are  also  link 
analysis  tools,  such  as  Netmap  (www.netmapanalytics.com)  or  Visual  Analytics 
(www.visualanalytics.com)  that  display  events  as  a  network  diagram,  or  graph,  of  objects 
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and  connections  between  objects.  Several  systems  exist  for  tracking  highly  connected 
information,  most  developed  for  intelligence  and  police  purposes  [Harries,  1999]. 
Analyst  Notebook  and  CrimeLink  provide  tools  for  organizing  and  arranging  fragments 
of  information  into  visualizations  intended  to  help  the  analyst  make  sense  of  the  complex 
relationships.  These  tend  to  be  one  dimensional  representations  that  show  either 
organizational  structures,  timelines,  communication  networks  or  locations.  In  each  case 
only  a  thin  slice  of  a  multidimensional  picture  is  portrayed.  Some  of  these  systems  are 
capable  of  using  animation  to  display  time.  Time  is  played  back,  or  scrolled,  and  the 
related  spatial  or  other  display  changes  to  reflect  the  state  of  information  at  a  moment  in 
time.  However  this  technique  relies  on  limited  human  short  term  memory  to  track  and 
then  retain  temporal  changes  and  patterns.  Another  visualization  technique  called  “small 
multiples”  uses  repeated  frames  of  a  condition  or  chart,  each  capturing  an  increment 
moment  in  time,  much  like  looking  at  sequence  of  frames  from  a  film  laid  side  by  side. 
Each  image  must  be  interpreted  separately,  and  side-by-side  comparisons  made,  to  detect 
differences.  This  technique  is  expensive  in  terms  of  visual  space  since  an  image  must  be 
generated  for  each  moment  of  interest,  which  can  be  problematic  when  trying  to 
simultaneously  display  multiple  images  of  adequate  size  that  contain  complex  data 
content. 

The  objective  of  our  research  has  been  to  develop  a  way  to  visualize  the  spatial  and 
temporal  dimensions  and  connectedness  of  the  problem  within  a  single  picture,  and  to 
make  it  feasible  for  use  by  the  commander  and  staff  within  the  battlefield. 

In  the  course  of  researching  concepts  and  developing  systems  for  ARDA’s  Geo-Spatial 
Intelligence  Information  Visualization  Program  (GI2Vis)  and  C2  users,  we  have 
developed  a  novel  visualization  technique  for  displaying  and  tracking  events,  people,  and 
equipment  within  a  combined  temporal  and  geospatial  display.  This  concept  has  been 
developed  into  a  demonstrable  prototype  called  GeoTime  in  order  to  determine  the 
potential  utility  by  both  intelligence  analysts  and  commanders. 


The  GeoTime  Visualization  Concept 

A  visualization  technique  was  needed  to  improve  perception  of  movements,  events  and 
relationships  as  they  change  over  time  within  a  spatial  context.  A  combined  temporal- 
spatial  space  was  constructed  in  which  to  show  interconnecting  streams  of  events  over  a 
range  of  time  in  a  single  picture.  Put  simply,  events  are  represented  within  an  X,Y,T 
coordinate  space,  in  which  the  X,Y  plane  shows  geographic  space  and  the  Z-axis 
represents  time  into  the  future  and  past  (see  Figure  1).  In  addition  to  providing  the  spatial 
context,  the  ground  plane  marks  the  instant  of  focus  between  before  and  after;  Events 
along  the  timeline  “occur”  when  they  meet  the  surface.  Events  are  arrayed  in  time  along 
time  tracks,  which  are  located  wherever  events  occur  within  the  spatial  plane. 
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Figure  1: 

Diagram  showing  how 
individual  frames  of 
movement  are 
translated  into  a 
continuous 
spatiotemporal 
representation 


Spatial  Time-Tracks 

Spatial  Time  tracks  make  possible  the  perception  of  where  and  when.  They  are  the 
primary  organizing  elements  that  support  the  display  of  events  in  time  and  space  within  a 
single  view.  Time-tracks  represent  a  stream  of  time  through  a  particular  Location  and  are 
represented  as  a  literal  line  in  space.  Each  unique  location  of  interest  will  have  one  spatial 
timeline  that  passes  through  it.  Events  that  occur  at  that  location  are  arranged  along  this 
timeline  according  to  the  exact  time  or  range  of  time  at  which  the  event  occurred. 


3D  Timeline  Possible  position 

of  future  event 


Future  (+Z  axis) 

Instant  of  Focus 
Past  (-Z  axis) 


Y  axis 


X  axis 


Possible  position 
▼  of  past  event 


Figure  2 

Diagram  of  3D  Timeline  configured  to  display 
past  as  down  and  future  as  up 


A  single  spatial  view  will  have  as  many  timelines  as  necessary  to  show  every  Event  at 
every  location  within  the  current  spatial  and  temporal  scope.  In  order  to  make 
comparisons  between  events  and  sequences  of  events  between  locations,  the  time  range 
represented  by  the  timelines  is  synchronized.  In  other  words  the  time  scale  is  the  same  for 
every  timeline. 


There  are  three  variations  of  Spatial  Timelines  that  emphasize  spatial  and  temporal 
qualities  to  varying  extents.  Each  variation  has  a  specific  orientation  and  implementation 
in  terms  of  its  visual  construction  and  behavior.  The  user  may  choose  to  enable  any  of  the 
variations  at  any  time  during  runtime. 


3D  Z-axis  Timelines 

3D  Timelines  are  oriented  normal  to  the  terrain  view  plane  and  exist  within  its  coordinate 
space  as  shown  in  Figure  3. 
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Figure  3 

Diagram  showing  how  3D  Timelines  pass 
through  terrain  locations.  3D  timelines  are  locked 
in  terrain  space  and  are  affected  by  changes  in 
perspective. 


3D  Viewer  Facing  Timelines 

3d  Viewer-facing  Timeline  are  similar  to  3D  Timelines  except  that  they  rotate  about  the 
instant  of  focus  point  so  that  they  always  remain  perpendicular  to  viewpoint  from  which 
the  scene  is  rendered.  This  is  shown  in  Figure  4. 


Figure  4 

Diagram  showing  how  viewer  facing  timelines  rotate  to 
face  the  viewpoint  no  matter  how  the  terrain  is  rotated 
in  3  dimensions. 


Linked  TimeChart  Timelines 


Linked  TimeChart  Timelines  are  timelines  that  connect  a  2D  grid  in  screen  space  to 
locations  marked  in  the  3D  terrain  representation.  As  shown  in  Figure  5,  the  timeline  grid 
is  rendered  in  screen  space  as  an  overlay  in  front  of  the  2D  or  3D  terrain. 


Location 


Diagram  showing  how  TimeChart  timelines  are  connected  to  terrain  locations. 
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The  GeoTime  Information  Model 

An  information  model  based  loosely  on  Davidsonian  semantics  [Davidson,  1980]  is  used 
to  support  the  representation  of  information  in  GeoTime.  The  following  objects  types  are 
employed  in  GeoTime. 

Entities  (people  or  things) 

represent  any  thing  related  to  or  involved  in  an  event  including  people,  objects, 
organizations,  equipment,  businesses,  observers,  affiliations  etc. 

Locations  (geospatial  or  conceptual) 

represent  a  place  within  a  spatial  context,  such  as  a  geospatial  map,  a  node  in  a  diagram 
such  as  a  flowchart,  or  even  a  conceptual  place  such  as  “OZ”. 

Events  (occurrences  or  discovered  facts) 

represent  any  action  that  can  be  described.  The  following  are  examples  of  events 
Bill  was  at  Toms  house  at  3  pm 
Tom  phoned  Bill  on  Thursday 
A  tree  fell  in  the  forest  at  4: 13  am,  June  3,  1993 
Tom  will  move  to  Spain  in  the  summer  of  2004 
Events  store  the  times  at  which  the  action  took  place. 

These  basic  information  types  are  combined  into  groups  using  Associations.  An 
association  is  an  information  object  that  describes  a  pairing  between  two  objects.  For 
example,  in  order  to  show  that  a  particular  entity  was  present  when  event  occurred,  an 
Association  is  created  to  represent  that  Entity  X  “was  present  at”  Event  A.  An  association 
also  exists  to  link  Event  A  to  a  particular  Location.  In  GeoTime,  groups  of  associated 
elements  have  been  defined  to  represent  certain  classes  of  occurrences  and  relationships. 
These  groups  have  specific  visual  expressions  and  interactive  behaviors  in  the  display. 
The  defined  groups  are  described  in  Figure  6.  A  variation  of  the  association  type  is  used 
to  define  a  subclass  of  these  groups  to  represent  user  hypotheses.  In  other  words,  groups 
can  be  created  to  represent  a  guess  or  hypothesis  that  an  event  occurred,  that  it  occurred 
at  a  certain  location  or  involved  certain  entities. 
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Group  Structure 

Event  1 

Location  A 


Group  Type  +  Sample  Application 

Observation  Group 

(An  Incident,  news  item  or  observation) 

1  Association 

• Event  1  occurred  at  Location  A 


Representation  Schematic 

e 


Location  A 


Location  A 


Observation  with  Actor 
(Involvement  in  an  incident,  observer) 
2  Associations: 

• Event  1  occurred  at  Location  A 
•Entity  X  present  at  Event  1 


8 


Location  A 


Event  1 


Location  A 


Event  2 


Location  B 


Soft  Vector  Group 

(Phone  Call,  email,  money  transfer) 

3  Associations: 

•Eventl  occurred  at  Location  A 
•Event  2  occurred  at  Location  B 
•Event  1  directed  at  Event  2 


Vector  Group  with  1+  Actors 
(Phone  Call,  email,  money  transfer) 

5  Associations:  Same  as  above  plus, 
•Entity  X  present  at  Event  1 
•Entity  Y  present  at  Event  2 


Event  1  Event  2 


Location  A 


Location  B 


Hard  Vector  Group 
(Documented  transport  such  as  air 
or  boat  travel) 

3  Associations: 

•Eventl  occurred  at  Location  A 
•Event  2  occurred  at  Location  B 
•Event  1  moves  to  Event  2 


Event  1 


Location  A 


Entity  X 


Location  B 


Event  1 

e- 


Entity  X 


Entity  Y 


Hard  Vector  Group  with  1+  Actors 
(Transport  of  a  person  or  thing) 

5  Associations:  Same  as  above  plus... 
•Entity  X  present  at  Event  1  (departure) 
•Entity  X  present  at  Event  2  (arrival) 


Actor  Relationship 

(Familial  or  employer-employee  relationships, 
organization  membership,  ownership) 

2  Associations: 

•Entity  X  member  of  Event  1 
•Entity  Y  member  of  Event  1 


-►9 


•  Location  B 


Location  A  C 


& 


Location 


5 


Event  1 

0-- 


Entity  Y 


Observation  Relationship 
(Indirect  involvement) 

1  Association: 

•Entity  X  member  of  Event  1 


/Location  A  •  x 

Location  B 


Figure  6  -Example  Entity  Activity  Groups 
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Information  Interaction  within  the  Spatio-Temporal  Workspace 

In  order  for  GeoTime  to  move  beyond  a  visualization  concept  to  a  usable  prototype, 
several  user  interactions  within  the  combined  spatio-temporal  viewer  were  necessary.  In 
addition  to  familiar  data  interactions  such  as  selection,  filtering,  hide/show  and  grouping 
that  operate  as  commonly  expected,  the  following  interactions  were  specifically 
developed  or  customized  to  work  within  the  GeoTime  environment. 

Temporal  Navigation 

The  Time  and  Range  Slider  is  a  linear  time  scale  that  is  visible  underneath  the 
visualization  representation.  This  control  contains  sub  controls/selectors  that  allow 
control  of  three  independent  temporal  parameters:  the  Instant  of  Focus,  the  Past  Range  of 
Time  and  the  Future  Range  of  Time.  The  time  range  visible  in  the  time  scale  of  the  time 
slider  can  be  expanded  or  contracted  to  show  a  time  span  from  centuries  to  seconds. 
Clicking  and  dragging  on  the  time  slider  anywhere  except  the  3  three  selectors  will  allow 
the  entire  time  scale  to  slide  to  translate  in  time  to  a  point  further  in  the  future  or  past. 

Simultaneous  Spatial  and  Temporal  Navigation 

Common  interactions  such  as  zoom-box  selection  and  saved  views  are  provided.  In 
addition,  simultaneous  spatial  and  temporal  zooming  has  been  made  possible  to  allow  the 
user  to  quickly  move  to  a  context  of  interest.  In  any  view,  the  user  may  select  a  subset  of 
events  and  zoom  to  them  in  both  time  and  space  using  the  Fit  Time  and  Fit  Space 
functions.  Within  the  Overlay  Calendar  views,  these  actions  happen  simultaneously  by 
dragging  a  zoom-box  on  the  time  grid  itself.  The  time  range  and  the  geographic  extents  of 
the  selected  events  are  used  to  set  the  bounds  of  the  new  view. 

Association  Analysis 

Functions  have  been  developed  that  take  advantage  of  the  association-based  connections 
between  Events,  Entities  and  Locations.  These  functions  are  used  to  find  groups  of 
connected  objects  during  analysis.  Associations  connect  these  basic  objects  into  complex 
groups  (see  Figures  6)  representing  actual  occurrences.  These  associations  can  be 
followed  from  object  to  object  to  reveal  connections  that  are  not  immediately  apparent. 
Association  analysis  functions  are  especially  useful  in  analysis  of  large  data  sets  where  a 
quick  and  efficient  method  to  find  and/or  filter  connected  groups  is  desirable.  For 
example,  an  Entity  maybe  be  involved  in  events  in  a  dozen  locations,  and  each  of  those 
events  may  involve  other  Entities.  The  association  analysis  function  can  be  used  to 
display  only  those  locations  on  the  visualization  representation  that  the  entity  has  visited 
or  entities  that  have  been  contacted.  Two  simple  association  analysis  functions  have  been 
implemented:  Expanding  Search  and  Connection  Search. 

1.  Expanding  Search 

The  expanding  search  function  allows  the  user  to  start  with  a  selected  object(s)  and  then 
incrementally  show  objects  that  are  associated  with  it  by  increasing  degrees  of  separation. 
The  user  selects  an  object  or  group  of  objects  of  focus  and  clicks  on  the  Expanding 
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Search  button  -  this  causes  everything  in  the  visualization  representation  to  disappear 
except  the  selected  items.  The  user  then  increments  the  search  depth  -  objects  connected 
by  the  specified  depth  are  made  visible  in  the  display.  In  this  way,  sets  of  connected 
objects  are  revealed. 

2.  Connection  Search 

The  Connection  Search  function  allows  the  user  to  connect  any  two  objects  by  their  web 
of  associations.  The  user  selects  any  two  objects  and  clicks  on  the  Connection  Search 
tool.  The  connection  search  algorithm  automatically  scans  the  extents  of  the  network  of 
associations  starting  from  one  of  the  objects.  The  search  will  continue  until  the  second 
object  is  found  as  one  of  the  connected  objects  or  until  there  are  no  more  connected 
objects.  If  a  path  of  associated  objects  between  the  target  objects  exists,  all  of  the  objects 
along  that  path  are  displayed  and  the  depth  is  automatically  displayed  showing  the 
minimum  number  of  links  between  the  objects. 


Application  of  GeoTime  to  Command  and  Control 

The  authors  envision  a  spectrum  of  applications  for  the  GeoTime  concept  in  Command 
and  Control  and  Intelligence.  These  applications  may  be  roughly  defined  by  a  space 
bounded  in  two  dimensions  -  data  character  and  operator  mode. 

Data  character  is  a  factor  of  the  source,  volume  and  connectedness  of  the  data.  At  one 
pole,  the  data  may  be  discrete  and  voluminous,  and  the  other,  focused  and  tightly 
interconnected.  Many  operator  modes  are  also  possible.  The  extremes  might  be  described 
as  information  consumer  vs.  information  creator.  Figure  7  shows  how  these  sets  of  poles 
can  define  a  range  of  C2  use-types.  Several  specific  use  cases  have  been  developed  in 
cognitive  task  analyses  and  reviews  with  analysts  and  commanders. 


Visual  History  of  Reports:  “Information  Consumer” 

The  GeoTime  display  can  be  used  to  get  an  instant  view  of  activity  at  any  time/space 
coordinate.  Within  a  C2  environment,  standard  activity  reports,  such  as  those  used  and 
produced  by  systems  like  ASAS  Lite,  CRIMELINK,  FBCB2,  MCS  and  others  can  be 
imported  and  translated  into  GeoTime  elements.  Over  the  course  of  weeks  or  months, 
hundreds  to  thousands  of  events  could  be  stored  and  reviewed  with  a  system  such  as 
GeoTime. 
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Tracking 

Insurgents 


Structured 
Long  term 
All  Source 
Vetted  Data 
User-guided  Input 
Tracking  over  time 
Structured  Archive 
High  Connectedness 


Figure  7. 

Envisioned  spectrum  of  C2 
applications  for  GeoTime 


Intell  Officer/ Analyst 
Multiple  Spatial  Contexts 
Create/Edit  data 
Track  people,  orgs,  events 
View  Query  Results 


User  in  the  field 
Ad-hoc  queries 
Localized  terrain  focus 
Track  reports  and  activity 


Unstructured 
Short  Term 
Single  Source 
Raw  Report  Data 
Data  from  feeds  or  DB 
Data  ages  quickly 
Low  Connectedness 


Insurgent  Tracking  Analysis:  i( Information  Creator  ” 

An  investigator,  such  as  an  intelligence  officer,  could  use  GeoTime  to  generate  an 
interactive  log  of  events  gathered  during  the  course  of  long-term  investigations  or 
tracking  task.  Existing  reports  and  query  results  are  combined  with  user  input  data, 
assertions  and  hypotheses.  The  investigator  can  replay  events  and  understand 
relationships  between  multiple  targets,  movements  and  the  events.  Patterns  of  travel, 
communications  and  other  types  of  events  can  be  understood.  Repetition,  regularity, 
bursts  or  pauses  in  activity  are  easily  apparent. 


Figure  8. 

Screenshot  of  the 
GeoTime  prototype 
showing  asymmetric 
events  and  entity 
tracking  over  time  in 
a  fabricated 
scenario. 


GeoTime,  Kapler,  Wright 


2004  CCRTS 


9 


GeoTime  For  C2 


Figure  9. 
Screenshot  of  the 
GeoTime  prototype 
in  calendar  mode 
showing  recent 
events  within  a 
localized  area. 


Evaluation  and  Conclusions 

Limited  demonstrations  of  GeoTime  running  against  unclassified  databases  of  simulated 
battlefield  events  and  analysts  training  scenarios  show  that  the  following  types  of 
information  are  quickly  revealed. 

What  significant  events  happened  in  this  area  in  the  last  X  days? 

Who  was  involved? 

What  is  the  history  of  this  person? 

Who  have  they  communicated  with? 

Where  might  they  have  met  with  other  people? 

Where  are  the  hot  spots? 

Has  this  type  of  event  occurred  here  or  elsewhere  in  the  last  Y  years 

Additional  GeoTime  evaluations  are  planned  in  the  near  future,  including  several 
technology  exploration  exercises  with  subject  matter  experts,  as  well  as  more  formal  task 
impact  and  evaluation  experiments. 

The  GeoTime  prototype  demonstrates  that  a  combined  spatial  and  temporal  display  is 
possible,  and  may  be  highly  effective  when  applied  to  analysis  of  past  and  future  events 
within  a  geographic  context,  such  as  an  area  of  operation.  Ongoing  research  will  include 
further  experimentation  to  explore  and  develop  the  full  potential  of  the  GeoTime 
visualization  tool  for  command  and  control. 
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GeoTime  For  C2 
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LSASO  Command  &  Control 

Challenges 


Track  people,  equipment,  organizations  and  their  activities  over 
time. 


•  Tracking  projects,  schedules.  Example:  reconstruction, 

•  I  n  situations  of  asymmetric  warfare,  stability  and  support 
operations  and  peace  support  operations,  tracking  relatively 
small  and  seemingly  unconnected  events  over  time  becomes  a 
means  for  gaining  insight  into  enemy  or  insurgent  behavior. 

•  Local  cultures,  characters,  organizations  and  their  behaviors  play 
an  important  part  in  planning  and  mission  execution. 

•  Small  attacks  come  from  unknown  assailants  over  long  periods 
of  time,  making  it  difficult  to  see  patterns. 

•  Tracking  relationships  between  events,  suspects,  known 
personalities  and  organizations. 

•  Information  obtained  is  piecemeal. 

•  The  key  dimensions  of  this  information  are  who,  what,  where, 
when,  how,  why  and  with-whom 
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C:  Event  Tracking 
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•  Current  C2  Systems  Track  events  separately  Tables,  Charts  and  Maps 

•  Difficult  for  users  to  get  a  complete  picture  of  events  across  all 
dimensions 

-  Space 

-  Time 

-  Sequence 

-  Relationships 

-  Event  Type 

-  Movement 

•  Each  dimension  shown  in  separate  views 

•  Reliance  on  human  memory  to  make  connections  and  coorelations 
across  these  dimensions  is  a  significant  cognitive  task 

-  Result:  analysis  suffers 

-  Proper  analysis  takes  a  great  deal  of  time 

-  Patterns  are  missed 
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Event  Tracking  over  Time: 

Examples 
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GeoTime  concept 


•  Visually  depict  events  over  time  and  space 
concurrently  -  x,y,t  space  (or  x,y,z-t  with 
Elevation  data) 

•  Develop  data  model  for  representing  Events 
and  Entities  involved  in  events 

•  Show  linkages  and  relationships 

•  Show  descriptive  event  icons 

•  Develop  the  visualization  interface  as  a  viable 
analyst  workspace  for  exploration,  deep 
analysis  and  presentation. 
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GeoTime  Visualization  Principle 


6 


•  Diagram  showing  how  individual  frames  of  movement 
are  translated  into  a  continuous  spatiotemporal 
representation 
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Time  Tracks 


Spatial  Time  tracks  make  possible 
the  perception  of  where  and 
when. 

Primary  organizing  device  for 
display  of  events  in  time  and 
space  within  a  single  view. 

Time- tracks  represent  a  stream 
of  time  through  a  particular 
Location  and  are  represented  as 
a  literal  line  in  space. 


3D  Timeline  - 
Future  (+Z  axis) 

Instant  of  Focus 
Past  (-Z  axis) 


Possible  position 
of  future  event 


Y  axis 


X  axis 


Possible  position 
of  past  event 
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Time  Track  Layouts 

•  Several  Time  Track  layout  schemes  have  been 
developed 

•  Each  seems  to  have  particular  properties  that 
emphasize  different  aspects  of  the  data 

•  User  preference  has  not  been  evaluated 
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3D  Layout 
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•  Emphasizes  spatial 
quality  of  data 


•  Temporal  comparison  is 
less  accurate 
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Viewer  Facing 


10 


•  Time  tracks  always 
oriented  to  viewer 


•  Temporal  comparison  is 
consistent  since 
perspective  distortion  is 
reduced 


9 
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Calendar  Modes 


•  Timeline  chart  overlaid 
and  connected  to  the 
geographic  dimension 

•  Emphasizes  temporal 
comparison 


Time 


▼V 


Location 

< - > 


Future  (+2  axis)  — > 

Instant  of  Focus  — > 
Past  (-Z  axis)  — > 

Timelines 

Event  Location 


Future  (+Z  axis) 
Instant  of  Focus 
^  Past  (-Z  axis) 
Timelines 


Proprietary  &  Confidential  ©  2004  Oculus  Info  Inc. 


Time 


oculus 


3D  Geospatial  Display 
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•  Where  it  is  relevant,  3D 
terrain  can  be  displayed 
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Timeline  Controls 


Playback 


Handle  Handle 


Past  Time 

(Above  ground) 


Instant  of  Focus 

(Ground  level) 


Future  time 

(Below  ground) 


NOTE:  The  direction 
of  future  and  past  can 
be  reversed  if 
preferred. 
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An  information  model  based  loosely  on  Davidsonian  semantics  [Davidson,  1980]  is 
used  to  support  the  representation  of  information  in  GeoTime.  The  following  objects 
types  are  employed  in  GeoTime. 

Entities  (people  or  things)  -  who? 

•represent  any  thing  related  to  or  involved  in  an  event  including  people,  objects,  organizations, 
equipment,  businesses,  observers,  affiliations  etc. 

Locations  (geospatial  or  conceptual)  -  where? 

•represent  a  place  within  a  spatial  context,  such  as  a  geospatial  map,  a  node  in  a  diagram  such 
as  a  flowchart,  or  even  a  conceptual  place  such  as  "OZ". 

Events  (occurrences  or  discovered  facts)  -  what,  how? 

•represent  any  action  that  can  be  described.  The  following  are  examples  of  events 
-Bill  was  at  Toms  house  at  3  pm 
-Tom  phoned  Bill  on  Thursday 
-A  tree  fell  in  the  forest  at  4: 13  am,  J  une  3,  1993 
-Tom  will  move  to  Spain  in  the  summer  of  2004 
•Events  store  the  times  at  which  the  action  took  place. 

Associations 

•An  association  is  an  information  object  that  describes  a  pairing  between  two  objects. 
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Taxonomy  of  Observation  Groups 
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Location  A 


Event  1  Event  2 

r  i 


Location  B 


Soft  Vector  Group 

(Phone  Call,  email,  money  transfer) 
3  Associations: 

•Eventl  occurred  at  Location  A 
• Event  2  occurred  at  Location  B 
• Event  1  directed  at  Event  2 


Location  B 


Event  2  I 

^  i 

/  Entity  Y 
Location  B 


Vector  Group  with  Actors 

(Phone  Call,  email,  money  transfer) 
5  Associations:  Same  as  above  plus.. . 
•Entity  X  present  at  Event  1 
• Entity  Y  present  at  Event  2 


e 


ft 

w 

Location  A 


0 


Location  B 


Event  1  Event  2 


Location  B 


Hard  Vector  Group 

(Documented  transport  such  as  air 
or  boat  travel) 

3  Associations: 

• Eventl  occurred  at  Location  A 
• Event  2  occurred  at  Location  B 
• Something  moves  to  Event  2 


Location  B 


Event  1 


Location  A 


Event  2 


Entity  X 


Hard  Vector  Group  with  Actors 

(Transport  of  a  person  or  thing) 

5  Associations:  Same  as  above  plus.. . 
•Entity  X  present  at  Event  1  (departure) 
•Entity  X  present  at  Event  2  (arrival) 


0  0 


Location  A  • 


Location  B 
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Taxonomy  of  Groups  Cont 
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Event  1 


Observation  Group 

(An  Incident,  news  item  or  observation) 
1  Association 

• Event  1  occurred  at  Location  A 


1 

Location  A 


Location  A 


Observation  with  Actor 

(Involvement  in  an  incident,  observer) 
2  Associations: 

• Event  1  occurred  at  Location  A 
•Entity  X  present  at  Event  1 


w 

Location  A 


Entity  X  Entity  Y 


Actor  Relationship 

(Familial  or  employer-employee 
relationships,  organization  membership) 
2  Associations: 

•Entity  X  member  of  Event  1 
•Entity  Y  member  of  Event  1 


Location  A 


Event  1 

■L 

Observation  Relationship 

0 - 

Li 

Entity  Y 

(Indirect  involvement) 

1  Association: 

•Entity  X  member  of  Event  1 

i  '' 

Location  A 

& 

Location  B 


s^i 

Location  B 
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Entity  Animation 
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Method  for  animating  entity  movement  between  events  during  time 
slider  interaction 


0 


Event  1 

Entity  X  observed  at 
Location  A  at  time  t 


0  Event  2 

Entity  X  observed  at 
Location  B  at  time  t+2 


1.  At  time  t,  Entity  X  is 
rendered  at  Eventl  (Location 
A) 


2.  At  time  t+1,  Entity  X  is 
shown  moving  between 
known  locations  (Eventl  and 
Event2) 


t  t+1  t+2 

i  i  i  i  Qc  i  i  i 

- ► 


3.  At  time  t+2,  Entity  X  is 
rendered  at  Event2  (Location 
B) 


The  Time  Slider  is  dragged  by  the  user  to  replay  the  sequence  of  events  from  time  t  to  t+2 
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Entity  Movement  & 
Communication 


•  Method  for  showing  communication  and  movement  events  between  Entities  over 
time. 

•  This  diagram  shows  a  static  view  of  Entity  ^making  3  phone  calls  to  Entity  Y 
from  3  different  locations  at  3  different  times. 

•  A  trail  is  drawn  to  show  previous  time  and  space  positions  for  Entity  X 


I  nstant  of 
F 


Entity  Y 
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Diagram  Space  Concept 
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•  Method  for  showing  Events  occurring  within  a  process  diagram-space  over  time 

•  This  diagram  shows  how  a  flowchart  or  other  graphic  can  be  used  as  a  spatial 
context  for  GeoTime  analysis.  I  n  this  case  the  object  (entity)  has  been  tracked 
through  a  production  process  to  the  final  stage. 
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Path  and  Link  Discovery 
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•  Tools  for  finding  connections 

-  Reveals  stories  I  ncrements  out  from  a  node 


-  Also  helps  to  de-clutter  display 


2  Types 

-  Radial  Connections 

•  Select  an  object  and 
incrementally  spread  outwards 
to  connected  events,  places  and 
targets 

-  Connections  Between  Points 

•  Select  any  2  objects  and  see  the 
paths  connecting  them 

•  Automatically  finds  the  shortest 
path 


l 
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Link  Discovery  Operations 
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•  Link  analysis  functions  use  the  structure  of 
associations  to  trace  links  between  objects 


Sequence  of  Linked  Objects 
showing  associations  in  red 


Entity  X 


Entity  Y 


Location  C 


E4 


Example  of  linked  entities 
as  they  would  appear  in  the 
display 


Sequence  of  Linked  Objects 


Entity  X 


...is  associated  with . . . 

0  Event  1 

...is  associated  with . . . 

0  Event  2 

...is  associated  with . . . 

Location  C  i 

...is  associated  with . . . 

0  Event  3 

...is  associated  with . . . 

0  Event  4 

...is  associated  with . . . 


Entity  Y 
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Descriptive  Event  I  cons 


,NH^| 


Business  Act 

Personnel  Act  (Fire,  Hire,  Promote,  etc) 
Businesses  Act  (Bankrupt,  Merge) 
Communication  Act 
Broadcast  (^) 

Mailing  [?] 

Meeting 

•  Net  Communication 

•  Email  Act 

News  Post 

Telephone  Call 
Telegraph 
Employment 

J  udicial  Act  (Conviction,  Execution,  tJA  O 
Imprisonment,  etc)  “ 

Movement 

Transfer* 

-  Travel  ^ 

Personal  Act  (Birth,  Death,  Wedding,  etc) 
Planning 

Police  Act  (Arrest,  Detainment,  Seizure,  J.' 
etc) 

Political  Act  (Election,  Impeachment, 
Recall,  etc)  . 


Production 

Construction  u 
Crop  Production 
Document  Production 
Manufacture  ^|| 
Terrorist  Act 

Assassination  ©1 
Attack  M/ 

Bombing  ^ 
Hijacking 
Hostage  Taking 
Kidnapping 
Surveillance 
Training 

Religious  Training  jv 
Secular  Training  u 
Weapons  Training 
T  rans  ^ 

Financial  Transfer  4* 

T  ransaction 

•  Loan  ^ 


n 


Purchase 
Theft 
T  ransfer* 


Sending 


T  ransportation 


Plus  Standard  2525b  Symbols 
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Demonstration 
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•  Afghanistan  Events 

•  Kapler  Family  Events 
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User  Task  Space 


-Tracking  Insurgents 
-Tracking  Equipment 


• 

Structured 

• 

Long  term 

• 

All  Source 

• 

Vetted  Data 

• 

User-guided  1  nput 

• 

Tracking  over  time 

• 

Structured  Archive 

• 

High  Connectedness 

All  Source  1C  Analyst 
Multiple  Spatial  Contexts 
Flat  Earth 
1C  Application 
User  enters  data  -  logbook 
Track  people,  money,  orgs 
View  Query  Results 


0) 
j  ^ 

Data  Character  £ 


\ 


£ 

o 


SASO? 


User  in  the  field,  I  ntel  officer 
Geographic  Emphasis 
3D  Terrain 
Military  operation 
Track  equipment  and  activity 
External  data  sources 
View  Query  Results 


•  Unstructured 

•  Short  Term 

•  Single  Source 

•  Raw  Data 

•  Data  from  feed  or  query 

•  Scanning 

•  Data  ages  quickly 

•  Low  Connectedness 


-  Operational 

-  Adhoc  Queries 
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Questions  GeoTime  Addresses 


What  significant  events  happened  in  this  area  in  the  last  X  days? 
Where  have  all  the  instances  of  this  type  of  event  occurred? 

I  s  there  a  pattern  to  these  events  in  time? 

Who  was  involved? 

What  is  the  history  of  this  person? 

Who  have  they  communicated  with? 

Where  are  the  hot  spots? 

Has  this  type  of  event  occurred  here  or  elsewhere  in  the  last  Y 
years 
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